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History 
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PerMIS’02  --  methods  to  perform  measurements 
PerMIS’03  —  large  experiments,  data  collection 
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FY03  Major  Experiments 


ARL/GDRS  -  Demo  III  TRL-6  experiments 
NIST  -  TRL-6  terrain  characterization  data  collection 
DARPA  -  PerceptOR  off-road  perception  experiments 
DARPA  -  MARS  Urban  Search  and  Rescue  Arenas 
Boeing/FCS  -  Integrated  Combat  Demonstration 
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FY03  Data  Collection 


Terrain  Characterization  Data 

High  resolution  LADAR  for  geometry 
High  performance  INS/GPS  for  position 
Instrumented  trailer  for  soil  mechanics 

Woods  and  fields,  desert  terrain,  snow,  urban 

Roads  and  traffic 

NIST  campus 
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TRL-6  Experiments 


Nov’02  -  Fort  Indiantown  Gap  -  Woods  and  fields 
Dec’02  -  Tooele  Army  Depot  -  Desert  sagebrush 
Jan’03  -  Fort  Indiantown  Gap  -  Woods,  trails,  snow 


Apr’03  -  Fort  Indiantown  Gap  -  Urban  environment 


Total  of  550  km  distance  traversed 

Over  90%  autonomous  operation  (distance,  time) 

Quantitative  measures  of  operator  interventions 


QENERAL 

□YNAMICS 

Robotic  Systems 
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DYNAMICS  TRL-6  Experiments  + 

Robotic  Systems  J- 

FCS  Integrated  Combat  Demo 


PerceptOR 

Off-Road  Perception 


Team  Raptor 


Team  Blitz 


NIST  •  Manufacturing  Engineering  Laboratory  •  Intelligent  Systems  Division 


PerceptOR  Goals 


•  Apply  perception  systems  to  dense  obstacle 
environments 


•  Minimal  sensor  development  (off  the  shelf) 


•  Develop  algorithms  while  testing  in  the  field 
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PerceptOR  Status 


•  Phase  II  concluded  -  DEC  02 

-  Four  field  tests  in  2002 

•  Phase  III  initiated 

-  Focus  on  Technical  Thrusts 

-  Two  Performers 

•  Team  Blitz  -  CMU  Prime 

•  Team  Raptor  -  SAIC  Prime 

-  (2)  Contract  tests  at  their  facility  in  1®*  half  of  2003 

-  (2)  Government  tests  in  2’“*  half  of  CY  2003 
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PerceptOR 

Ph  II  Communications 


Conventional 
Teleop  >  2  Mb/s^ 


NIST  •  Manufacturing  Engineering  Laboratory  •  Intelligent  Systems  Division 


DARPA  MARS 


Urban  Search  &  Rescue  Testbed  ' 

•  3  Robotic  USAR  Competitions 

—  RoboCup  2003 ,  Rome,  Italy 

—  IJCAI  2003,  Acapulco,  Mexico 

—  U.  S.  RoboCup  Rescue  Open  2003,  CMU 

•  Scored  competitions 

•  Analysis  of  system  performance 


Rescue 


Cup 
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IJCAI/AAAI  ARENAS 

(MEXICO,  2003) 
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IJCAI/AAAI  ROBOTS 

(MEXICO,  2003) 
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ROBOCUP  ROBOTS 

aXALY,  2003) 
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USAR  ROBOT 
PERFORMANCE  METRIC 
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America 


German  Oti 


Japan  Open 


..obooupRescue 

LISBON,  PORTUeAL 


AAAI  Confere 


CALIFORNIA.  USA 


DTTIQNS 


PREVIOUS  COMPETITIONS 


AAAI  Conference,  2000 

AUSTIN,  TEXAS,  USA^f 

IJCAI/AAAI  Conference  2001 

SEATTLE,  WASHINTON,  USA 

\ 

RoboCupRescue  2002 

FUKUOKA  JAPAN  * 

AAAI  Conference  2{fe2 

EDMONTON,  ALBERTA,  CAIMda 

American  Open  200^'  , 

PENNSYLVANIA^  USA  jU  > 

Japan  Open  2003 

NI]feATA,  JAPAN  i- 

RoboCupRescue  2003  i  ^ " 

PADUA,  ITALY 

IJCAI/AAAI  Conference  2j#03 

ACAPULCO,  MEXICO 


NIST 

MARYLAND,  USA  (2000) 

Museum  of  Emerging  Science 

TOKYO,  JAPAN  (2002) 

/ 

Carnegie  Mellon  University 

PENNSYLVANIA,  USA  (2003) 

Institute  Sup4riore  Anticendi 

ROME,  ITALY  (2Cro3)»^ 


~^pSTEIiIN&  collaboration  THROUGH  STANOAR&S^ 


HIGH  RESOLUTION  LADAR 

FOR  TERRAIN 
CHARACTERIZATION 

NIST  HMMWV 
at  Tooele  Army  Depot 
Dec.  9-13,  2002 


Mike  Shneier,  Tsai  Hong,  Tommy  Chang, 

Harry  Scott,  Steve  Legowik,  Gerry  Cheok,  Chuck  Giauque 

Intelligent  Systems  Division 
National  Institute  of  Standards  and  Teehnology 
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Terrain  Characterization  Testbed 
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Black  Course:  Tooele  Army  Depot 


NIST 


GENERAL 

nVNAMICS 

Robotic  Systems 
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Ravine 

200x200  m,  1 6cm  grid  size,  Overhead  View 


iMisr 


Elevation  (m) 


Estimated  Pitch  and  Roll 
for  Ravinel  200  x  200  m 


•  Surface  contour  intervals  set 
@  0.2  m. 

approximately  (0,  60)  and 
proceeds  to  the  right. 

•  Note  gap  in  the  path  data 
leads  to  a  gap  in  the  roll  and 
pitch  plots. 


Pitch  angle  (degrees)  along  path  (+  up,  -  down) 


E-STOP  94  on  the  Dam 


NIST 
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uTm 


E-Stop  94  on  the  Dam  kNisr 


View  Point  2  LADAR  intensity  image 


CCD  Camera  Image  composite 


E-Stop  94 


NIST 


View  Point  2  Looking  Northward 

X-section  showing  slopes  of  dam 
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Ground  Slope  on  the  Dam 
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Terrain  Analysis  Using  High 
Performance  INS/GPs 


ARL/NIST  HMMWV 
at  Tooele  Army  Depot 
Dec.  9-13,  2002 

Harry  Scott,  Steve  Legowik,  Mike  Shneier,  Tsai  Hong,  Tommy  Chang, 

Harry  Scott,  Steve  Legowik,  Gerry  Cheok,  Chuck  Giauque 

Intelligent  Systems  Division 
National  Institute  of  Standards  and  Teehnology 
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TARDEC  Instrumented  Trailer 
for  Measurement  of  Soil  Mechanics 

•  Features  of  TARDEC  Trailer: 


-  Load  cell/strain  gage  on  trailer  tongue  with  signal  conditioning 

-  Electronic  braking/micro-processor  controlled  (for  rolling 
resistance  and  ground  friction  measurements) 

-  Variable  load  on  trailer  axle  (bolted  steel  plates  -  matches  XUV 
loading ) 

-  XUV  tires  and  12”  ground  clearance  on  trailer  (highly  mobile) 


Top  View 


Isometric  View  Trailer  Hitch 
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TARDEC  Instrumented  Trailer 
at  Tooele  Army  Depot 
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Terrain  Characterization 


Active  Volumetric  Mapping 


Sebastian  Thrun,  Geoffrey  Gordon  Gates  Hall,  Stanford  U 
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Technology  Requirements 
for  Autonomous  On-Road  Driving 


Analysis  of  tasks 

what?  when?  why?  how?  where? 


Decomposition  into  levels  based  on 

range  and  resolution  in  time  &  space 

Analysis  of  knowledge  requirements 

geometry,  dynamics,  entities,  events,  situations 


Analysis  of  sensor  requirements 

field  of  view,  resolution  in  time  and  space 
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Driving  Task  Analysis 


1)  Analyze  Autonomous  Driving  Tasks  &  Deveiop  a  Task  Architecture 


DOT- 

Manual 


T ask  Analysis 

+ 


Gaze  Control 


Road  Driving  Data 
Representations 


LaneSegment 

Trajectory 

Generator 


Selects  Destination 

2  hour  plan 

Selects  Road  Segments 

10  min  plans 

Selects  Lane 

1  min  plans 

Selects  Pos  &  Speed  In  Lane 
10  sec  plans 

Creates  Trajectory 

1  sec  plans 


steer 

Speed 

Servo 

Servo 

Servos  Steer  &  Speed 
100  ms  plans 
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Driving  Task  Analysis 


2)  From  task  architecture,  derive  finite-state  automata 
that  encode  task  behaviors 


World  Model  Value  Behavior 

Situation  Judgment  Generation 
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Driving  Task  Analysis 


3)  From  FSMs,  derive  decision 
criteria  and  Vaiue  Judgments 


World  Model  Value  Judgment 
Situation 


Pass  on  Two  Lane  Road 


NewPlan 

S1  SetPassOnTwoLaneRoadContextQ 
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LookingToPass 
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ic_ChangeTo_LeftLane 
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S3  AdjustPassingParamsQ 

ic_FollowLane 

PassingVehicle 

S3  ClearOfPassedVehicle 
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S4  ReturnedToLane 

SO  Done  SetNormalDriveParamsO 
ic_Follow_Lane 

FinishedPassManeuver 
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Plan 
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Driving  Task  Analysis 


LegalToPass 


WeatherNotObscuringPassZone 

RoadSplashNotSignifcant 

WindsNotSignificant 

RoadSurfaceNotToSlipperyToPass 

RoadSurfaceSuitableToPass 

OwnVehicleCapableToPass 


EnvironmentSafeT  oPass 


NoHillBlockingSightInPassZone 
NoCurveBockingSightInPassZone 
NoVehicleInFrontAttemptingLeftTurn 
NoVehicleEnteringRoadInPassZone 
VehInFrontNotBlockingSightInPassZone 
NoPostalVehicleOrDeliveryVehicleMakingStops  \ 
NoPedestrianOnRoadSideInPassZone 
SufficientReturnSpaceInFrontAfterPass 
VehicleInFrontDrivingNormally 
VehicleInFrontNotAttemptingToPass 
NoPersonOnBikeInPassZone 
NoVehicleOnRoadsideReadyToComeIntoLane 
NoActiveEmergencyVehiclesInFront  / 


SituationInF  rontOKtoPass 


NoConstructionInPassZone 
NoTransitOrSchoolBusStopping 
NoPassZone-NotIn  Effect 
LaneMarkingsAllowPass 


SituationInBackOKtoPass 


NoIntersectionsInPassZone 

NoRailroadXInPassZone 


OnComingTrafficOKtoPass 


World  Model 
Situation 


4)  Derive  dependencies  on 
World  Model  Situations 
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SufficientReturnSpace 


S4  ReturnedToLane 


Plan 
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Behavior 

Generation 
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Driving  Task  Analysis 


NoConstmctioninPassZone  5)  Derive  World  Model  Knowledge  requirements 

NoTransitOrSchoolBusStopping 

NoPassZone-NotInEffect  V-  LegalToPass 
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Driving  Task  Analysis 


6)  Define  Entities, 
Events, 
Attributes, 
Resoiutions, 
and  Tolerances 

CrossBuck(pos) 

Lights(pos,  state) 

Crossing  Gate(pos) 
Signs(pos,  facing-dir, 
text  and  graphics) 
Tracks(pos,  dir) 

Train(pos,  dir) 

Lanes(pos,  dir,  width, 
curvature) 

PassingZone(veh  speeds, 
safety  buffer,  accel) 
eg.  All  attributes  must 
be  recognizable  out 
to  600  to  800  feet 


NoConstructionInPassZone 

NoTransitOrSchoolBusStopping 
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/  -  ^ 
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World  Model 
Entities,  Events,  and 
Attributes 


World  Model 
Situation 


Value  Judgment 


Behavior 

Generation 
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Driving  Task  Analysis 


7)  Define  Requirements 
for  Perception 


Sensory 

Processing 


World  Model 
Entities,  Events,  & 
Attributes 


World  Model 
Situation 


Value  Behavior 

Judgment  Generation 
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Analysis  of  knowledge  requirements 


Knowledge  of  geometry  and  dynamics 
where  are  things? 
where  are  they  going? 
what  size,  shape,  feel? 
what  relationships  to  other  things? 

Knowledge  of  class  membership 
what  are  things? 
what  are  their  characteristics? 
how  can  they  be  expected  to  behave? 
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Image  processing  requirements 


A  LADAR  intensity  image  of  cars  in  parking  lot 
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Image  processing  requirements 


Segment  object  of  interest 


800 

900 

1000 


Max  range  =  10.977  meters 
Min  range  =  6.85  meters 
Minimum  range  at  (693,  372) 
Leftmost  point  =  (613,  232) 
Rightmost  point  =  (653,  506) 
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Image  processing  requirements 


Objects  on  a  virtual  road 
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Image  processing  requirements 


Segmented  objects  on  virtual  road 


Image  processing  requirements 

Map  View  of  Cars  Segmented  from  Pavement 

100 

200 

300 

400 

500 

600 

700 

800 

900 

1000 

100  200  300  400  500  600  700  800  900  1000 


Flash  LADAR  128  x  128  range  pixels 


□ZIOI] 
c  ]  n  c  ] 


:uqd  d 


dLidn 


dUdd  dH 
nriHrinn 


□  dqEj 


d  □  u  d 


rdddd 


L3dp 


□  cidd 


L4duqu 


NIST  •  Manufacturing  Engineering  Laboratory  •  Intelligent  Systems  Division 


Flash  LADAR  128  X  128  range  pixels 


NIST  •  Manufacturing  Engineering  Laboratory  •  Intelligent  Systems  Division 


RealA^irtual  Environment 
for  Software  Development 


Measures  as  Specifications 


Performance  requirements  =>  performance  measures 
Performance  measures  =>  design  specifications 
Design  specifications  =>  technology  requirements 
Technology  requirements  =>  research  goals 
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Example 

Performance  requirement  =  pass  on  two  lane  road 

Performance  measures  =  accelerations,  clearance, 
perception  range,  accuracy,  &  latency 

Design  specs  =  sensor  range,  resolution,  algorithms 
for  perception  &  control,  computing  power 

Technology  requirements  =  LADAR  specs,  camera 
technology,  INS/GPS,  computing  hardware 

Research  goals  =  lasers,  detectors,  speeds,  materials 
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Example  2 

Performance  requirement  =  enter  thru  hole  in  ceiling 

Performance  measures  =  dimensions,  speed,  mobility, 
mapping,  target  detection,  operator  workload 

Design  specs  =  vehicle  performance,  sensor  specs, 
perception  &  control,  computing  power 

Technology  requirements  =  tether  control,  camera 
technology,  3-D  localization  and  mapping 

Research  goals  =  cable  dynamics,  kinematics, 
materials,  image  registration 
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Summary  and  Conclusions 


We  have  begun  to  move  from  theory  to  experiment 
Several  major  experiments  have  been  accomplished 
Performance  measures  are  being  developed 
Performance  measures  are  driving  specifications 
Specifications  are  driving  technology  development 
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